Skip to content

feat(coding-agent): ACP semantic-edges-v1 provenance producer - #1885

Merged
sethkarten merged 16 commits into
mainfrom
feat/acp-lineage-v1
Sep 3, 2026
Merged

feat(coding-agent): ACP semantic-edges-v1 provenance producer#1885
sethkarten merged 16 commits into
mainfrom
feat/acp-lineage-v1

Conversation

@snimu

@snimu snimu commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Which model call caused what? Today that answer can only be guessed from transcripts, and transcripts mislead in exactly the places that matter: a retried call leaves duplicate-looking entries, a compaction severs the chain it was supposed to summarize, and a subagent's output records nothing about the parent call that spawned it. Downstream eval and RL pipelines run reconstruction heuristics over transcripts to recover causality the harness already knew. This PR makes the coding agent state it instead: a provenance producer that durably records, per session, which call caused what — across retries, compactions, and subagent spawns.

  • Every session-history model call (turn calls and compaction-summary calls) carries one opaque correlation ID, sent as both X-ACP-Model-Request-ID and Idempotency-Key (same value).
  • A per-session append-only ledger, semantic-edges.jsonl beside the session's RLM artifacts, records session_registered, request_started, request_finished, request_failed, compaction_begun, compaction_finished, and child_returned events — written before the action they describe, so a provider can never observe a request ID that is absent from the ledger.
  • The edge set (continuation, subagent_call, subagent_return, compaction) is derived on demand by a pure, order-independent fold over any set of ledgers; edges publish only after their target request commits.
  • Retry identity: one ID is reused across provider-SDK retries for free, and across session-level auto-retries only when the request body is identical (eager sha256 over provider, model, system prompt, and messages), so one Idempotency-Key never covers two different bodies. Measured on this machine: ~0.4 ms per call at a 50k-token context (~420 KB serialized), ~1.0 ms at 200k (~1.6 MB) — negligible against the wire call it fronts.
  • Spawn ancestry (parent session, spawning request ID) propagates through all three subagent hosts (inline, runtime, daemon), snapshotted synchronously at the spawn entry point and restored across parent resume. A child is attributed to the parent request only when the spawn arrives during an active run; an out-of-band spawn records no spawned_by_request_id — an absent edge beats a wrong one.
  • Crash tolerance: torn final ledger lines are repaired on first append only, reads never mutate the file, and an in-flight request at a torn tail derives zero edges. Split-turn summary slices settle at the compaction outcome, not at wire success — a failed or cancelled compaction commits no summary request, so the next turn's continuation edge always points at the pre-compaction request.

Scope: calls outside session history — side questions, /refine planning and auto-review, branch summarization, the daemon session summarizer — deliberately carry no correlation. They produce no message node downstream, the contract treats uncorrelated calls as ordinary traffic (ModelCall.acp=None), and recording them would corrupt the session's continuation chain.

The wire contract (ai.prime.acp/semantic-edges-v1) is pinned externally by the consumer in the verifiers repo (verifiers PR #2449, merged) and the producer side in nano-rlm (PR #153). This PR produces the ledger and the derivation; existing provenance (X-Trace-Id, agent traces) is untouched.

Validation: new suites semantic-edges (52 tests: recorder, fold, wrapper) and agent-session-semantic-edges (22 tests: headers, retry identity, compaction, spawn ancestry across hosts, child return, racing-slice failure, out-of-band spawn) — the racing-slice and out-of-band pins verified fail-unfixed on the pre-fix code. Revalidated after a conflict-free merge of current main (v0.9.1): daemon-mode (184) plus the touched agent-session, compaction, retry, recursion, and side-question suites — 600 tests, 0 failures. Mutation-tested: 35 mechanics mutants (ledger ordering, retry identity, ancestry, torn-tail, epoch gating) plus 12 semantic-edge mutants, all killed by named tests. Full npm run check (biome, tsgo, installer render, browser smoke) green on the merged tree.

Linear: RES-1245 https://linear.app/primeintellect/issue/res-1245

snimu added 7 commits August 28, 2026 10:30
One opaque request ID on the wire (X-ACP-Lineage-Request-ID +
Idempotency-Key, minted before the call and stable across retries of the
same call), a durable append-only lineage.jsonl ledger per agent session
beside its artifacts, context epochs driven by compaction outcomes,
cross-process ancestry for spawned subagents, and deriveLineageManifest
folding a session tree's ledgers into the verifiers lineage-v1 manifest.
Derivation only; nothing publishes or reads the ledger yet.
… ledger repair

Review round 2 on the lineage-v1 producer: retry Idempotency-Key reuse now
requires a byte-identical body (sha256 of model + system prompt + messages)
instead of a shape proxy; spawn ancestry is snapshotted synchronously at the
spawn entry point and survives resume via ledger replay; child terminal
status is recorded once at actual release (reusable children stay running
across follow-up runs, startup failures record failed); ledger construction
never mutates the file (torn-tail repair deferred to first append) and only
a malformed unterminated final line counts as torn; the completed-compaction
ledger event now precedes the transcript commit. Adds host pass-through,
real-summary-call wire, negative validator calibration, replayed-status,
double-wrap, and failed/cancelled mapping coverage.
Round 3 on the lineage-v1 producer. The retry body hash is now computed
eagerly at request creation, before the wire call, so mutating the live
message objects between the failed call and the retry park can no longer
alias two different bodies under one Idempotency-Key (TOCTOU regression
test included; the lazy-thunk indirection is gone). Test hardening from
the mutation campaign: hash keyed on provider/model/systemPrompt each
proven independently; undefined-vs-undefined parked hashes never match;
readLineageLedger proven read-only; torn-tail repair proven single-shot;
spawn-ancestry snapshot proven to precede preflight awaits; the completed
compaction ledger event proven durable before (and exactly once despite
a failing) transcript commit. The invariant port checks compactions
before requests and the negative calibration table asserts the named
error per row, adding duplicate-context, session-owner, and per-predicate
isolation rows.
readLineageLedger is proven to never open the ledger for writing (a
missing path must throw ENOENT and stay missing; a zero-byte append-open
would create it), and the calibration table gains a row where a fully
valid child compaction is named by a root request, so only the
request-side compaction owner check can reject it.
…edges-v1

The ecosystem replaced lineage-v1 (nano-rlm #153, verifiers #2449): one
renamed correlation header (X-ACP-Model-Request-ID + Idempotency-Key) and
sparse commit-gated semantic edges instead of a full session manifest.
The durable per-session JSONL ledger keeps its hardened mechanics
(ledger-before-wire, torn-vs-corrupt tails, repair-on-first-append,
read-path purity, replay idempotence, eager body-hash retry identity,
synchronous spawn snapshots) and now records request outcomes: the
streamFn wrapper commits or fails each request when its stream resolves,
compaction summaries commit around the real wire call only, and a parent
claims a child_returned with the child's last committed request at the
success point. deriveSemanticEdges is a pure order-independent fold to
continuation/subagent_call/subagent_return/compaction edges: edges
materialize only when their target commits, failed requests return their
inbound edges to the session's pending set (prime-agent has no prompt
rollback, so they attach to the next commit), spawn edges defer to the
child's first committed request, and a completed compaction suppresses
the summary's own continuation. Deleted: context epochs, session status
recording, depth in ledger events, the manifest deriver, and the
lineage-v1 validator port with its calibration table.
Two isolated compaction fold gates (a committed summary followed by a
later commit, and a failed compaction after a committed summary, each
producing no compaction edge), distinct wrapper outcome cases including a
rejected stream promise and an aborted final message (both must fail the
request), and the wire header asserted as the literal string
X-ACP-Model-Request-ID so a renamed production constant cannot pass.
Aligns with nano-rlm a1d7856: a failed child still returns an error
outcome the parent consumes, so the failure settle point now records
child_returned with the child's last committed request. Cancelled runs
and zero-commit children stay silent; the returned-once guard is
unchanged. Tests cover the failed-with-commits return edge, failed
zero-commit silence, and cancelled-with-commits silence.
Comment thread packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/core/semantic-edges.ts
Comment thread packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
…lt paths

Review-bot round: the retry body hash now covers tools and the
request-shaping stream options (reasoning, thinking budgets, temperature,
max tokens, service tier); parking is forfeited entirely when a
before_provider_request hook is registered, since payload hooks rewrite
the wire body after the hash point; ledger write failures in the stream
outcome observer and in compaction error paths are contained with a
one-line warning instead of crashing or masking the original error (the
completed-compaction ID is marked consumed before its write so a failed
write propagates untouched); and side questions now use the unwrapped
inner stream function, keeping their calls out of the session ledger and
the continuation chain.
Comment thread packages/coding-agent/src/core/agent-session.ts
Split-turn compactions send two summary calls with different bodies; one
shared Idempotency-Key gets rejected (or replayed) by key-honoring
interceptors. compact() now runs every summary call through a host
runner that mints its own request ID, headers, and finish/fail outcome,
and the edge fold tracks all summary slices of a compaction, keying the
compaction edge off whichever slice is the session's last commit. Earlier
slices contribute ordinary continuation edges.
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
Comment thread packages/coding-agent/src/core/semantic-edges.ts Outdated
Comment thread packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
Comment thread packages/coding-agent/src/core/agent-session.ts
…wing

Review-bot round 2, one mechanism for four findings: the first failed
ledger operation (read at construction or any append) permanently
disables the recorder with a single warning; every write becomes a no-op
and the streamFn wrapper and compaction runner stop emitting request IDs
on the wire, preserving ledger-before-wire instead of weakening it. This
removes the scattered try/catch layers (recordOutcomeSafely and both
compaction-path guards are deleted) and means an unwritable ledger can no
longer break compaction, rewrite a successful child run to error, or fail
a compaction whose summary succeeded. Appends are now durable before
in-memory commit state advances, so a failed write never leaves a
claimable commit. Aborted compaction summaries record request_failed
rather than committing partial text into the chain. Compaction summary
slices claim no pending edges and return none on failure (documented
divergence from nano-rlm): pending defers to the post-compaction turn,
the request that actually consumes subagent results.
Comment thread packages/coding-agent/src/core/agent-session.ts
snimu added 3 commits August 29, 2026 10:29
…slice

A completed compaction as the session's final activity permanently lost
deferred pending edges (subagent_return, reclaimed continuations): the
slices claimed nothing and no post-compaction turn ever arrived. The fold
now flushes still-pending edges to the compaction's last-committed slice
at completion — the same request that sources the compaction edge — so
pending always lands on a committed request whether or not the session
continues, restoring nano-equivalent single-call semantics while staying
deterministic for split turns. Failed or cancelled compactions still
leave pending for the next turn.
…uations

The terminal flush appended pending edges after the last summary slice
had already emitted its regenerated continuation, so a reclaimed
continuation from the same source derived twice — duplicate edges the
consumer rejects. The flush now applies nano's source-only suppression:
a pending edge from X removes the slice's generated continuation from X
regardless of the pending edge's type, then pending flushes once.
# Conflicts:
#	packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
Comment thread packages/coding-agent/src/core/agent-session.ts

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 3e473f3. Configure here.

Comment thread packages/coding-agent/src/core/agent-session.ts
…mits and gate spawn attribution on an active run

A split-turn summary slice that succeeded on the wire no longer publishes request_finished at stream resolution: slices settle at the compaction outcome, so a failed or cancelled compaction leaves no committed summary request and the next turn's continuation edge stays on the pre-compaction request. Spawn attribution now requires an active agent run; an out-of-band spawn records no spawned_by_request_id instead of a stale one.
@sethkarten
sethkarten self-requested a review September 3, 2026 15:22
@sethkarten
sethkarten merged commit 1768ace into main Sep 3, 2026
26 checks passed
@sethkarten
sethkarten deleted the feat/acp-lineage-v1 branch September 3, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants